Amendments to the Claims: 



This listing of claims will replace all prior versions, and listings, of claims in the 
application: 

Listing of Claims: 

Claim 1 (Currently Amended) An apparatus for associating interactive data representative 
e fwith an object with in a digital video programme including means for providing a digital video 
programme having plural individual frames at least some of which incorporate said object, 
means for parsing the video programme by identifying separate shots in the video programme to 
produce an edit list, means for identifying shots containing related content to form a sequence of 
shots containing related content, means for selecting at least one key- frame within each sequence 
of shots, means for selecting said object within the key-frame with which the interactive data is 
to be associated, means for extracting attributes of the object from the key- frame, means for 
associating the interactive data with the object in the key-frame, means for utilising the attributes 
of the object for tracking the object through the sequence of shots, whereby said interactive data 
is associated with the object in the sequence of shots and said interactive cont e nt data is 
embedded with data representative of said object in a data sequence. 

Claim 2 (Original) An apparatus as claimed in claim 1, wherein the means for identifying shots 
containing related content to form a sequence of shots containing related content includes means 
for producing a hierarchy of groups of shots. 

Claim 3 (Previously Presented) An apparatus as claimed in claim 1, wherein said means for 
parsing include means for inputting criteria to be used to recognise a change of shot. 

Claim 4 (Previously Presented) An apparatus as claimed in claim 1, wherein the means for 
extracting attributes of the object includes means for isolating the object within a boundary 
formed on the frame, means for performing edge detection within the boundary to identify and 
locate edges of said object, and storing means for storing a geometric model of said object. 
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Claim 5 (Previously Presented) An apparatus as claimed in claim 1, wherein said means for 
extracting attributes of said object also includes means for recording at least one of the attributes 
of shape, size, position, colour, texture, intensity gradient of said object, and time series statistics 
based on said attributes. 

Claim 6 (Previously Presented) An apparatus as claimed in claim 1, wherein said means for 
extracting attributes of said object includes means for comparing said attributes of said object 
with attributes of objects previously stored to determine whether the object is distinguishable 
therefrom, and when said object is determined not to be distinguishable, providing means for 
redefining the object. 

Claim 7 (Previously Presented) An apparatus as claimed in claim 1, wherein said means for 
extracting said attributes includes means for comparing the location in the frame of said object 
with the location of objects already stored for that frame to determine whether that object is 
distinguishable therefrom, and where the location of said object is not distinguishable from the 
location of another object providing means for assigning rank to the objects to determine which 
object will be associated with that location. 

Claim 8 (Previously Presented) An apparatus, as claimed in claim 1, wherein the means for 
utilising the attributes of the object for tracking the object includes means for updating the stored 
attributes of the object change from frame to frame. 

Claim 9 (Previously Presented) An apparatus as claimed in claim 1, wherein said tracking 
means utilising the attributes of the object for tracking the object includes plural algorithm means 
for calculation of independent tracks of objects for use depending on the visual complexity of a 
sequence to automatically track objects in different types of visual environment. 

Claim 10 (Previously Presented) An apparatus as claimed in claim 1, wherein said tracking 
means for utilising the attributes of the object for tracking includes means for converting all the 
frames to be tracked to a low-level representation, means for determining the position of each 
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object in the frames by minimising a distance measure to locate each object in each frame, means 
for processing the positions of said object to smooth over occlusions and the entrances and exits 
of objects into and out of said frames, and means for reviewing the object within a tracked 
sequence and for correcting the location attributes of any misplaced objects. 

Claim 1 1 (Previously Presented) An apparatus, as claimed in claim 1 , wherein the means for 
associating interactive data with the object in the key- frame includes means for providing a 
database of different types of data including one or more of URLs, HTML pages, video clips, 
audio clips, text files and multimedia catalogues, and means for selecting said interactive content 
data from the database to associate with said object. 

Claim 12 (Previously Presented) An apparatus, as claimed in claim 1, wherein the means for 
associating interactive data with the object in the key-frame produces said data sequence using 
means for determining whether the embedded interactive content data is frame synchronous data 
associated with object positions, shapes, ranks and pointers in a frame, or group-synchronous 
data associated with all the objects in a group, or is data to be streamed just in time, wherein 
means are provided for associating frame synchronous data with the corresponding frame, means 
are provided for associating group synchronous data with the frame at which a group changes, 
and means are provided for streaming just in time data to a user before it is required to be 
associated with the corresponding objects. 

Claim 13 (Previously Presented) An apparatus as claimed in claim 1, wherein means are 
provided to associate different interactive content data with respectively different objects. 

Claim 14 (Currently Amended) A method for associating interactive data repres e ntative of with 
an object with in a digital video programme including the steps of: 

a) providing a digital video programme having a plurality of individual frames at least some of 
which incorporate said object with which interactive data is to be associated, 

b) parsing the video programme by identifying separate shots in the video programme to produce 
an edit list, 

c) identifying, from the edit list, shots containing related content to form a sequence of shots 
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containing related content, 

d) selecting at least one key-frame containing the object from within the sequence of shots, 

e) locating said object within the at least one key-frame, 

f) extracting attributes of the object from the at least one key-frame, 

g) associating the interactive data with the object in the at least one key-frame, 

h) tracking the object through the sequence of shots utilising the attributes of the object, 

i) associating said interactive data with the object in frames in the sequence of shots, and 
j) embedding said interactive data with data representative of said object in a data sequence 
representative of the digital video programme. 

Claim 15 (Original) A method as claimed in claim 14, wherein step b) includes the step of 
inputting criteria (420) to be used to recognise a change of shot. 

Claim 16 (Previously Presented) A method as claimed in claim 14, wherein step c) includes the 
step of producing a hierarchy of groups of sequences of shots. 

Claim 17 (Previously Presented) A method as claimed in claim 14, wherein step e) includes the 
steps of: isolating the object (600) within a boundary (510) formed on the frame, performing 
edge detection within the boundary to identify and locate edges of said object, and step f) 
includes storing a geometric model of said object. 

Claim 18 (Previously Presented) A method as claimed in claim 14, wherein step f) includes the 
step of recording at least one of the attributes of shape, size, position, colour, texture, intensity 
gradient of said object, and time series statistics based on said attributes. 

Claim 19 (Previously Presented) A method as claimed in claim 14, wherein step f) includes the 
step of comparing said attributes of said object with attributes of objects previously stored to 
determine whether the object is distinguishable therefrom, and when said object is determined 
not to be distinguishable, the step of re-defining the object. 

Claim 20 (Previously Presented) A method as claimed in claim 14, wherein step f) includes the 
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step of comparing the location in the frame of said object with the location of objects already 
stored for that frame to determine whether that object is distinguishable therefrom, and where the 
location of said object is not distinguishable from the location of another object, the step of 
assigning rank to the objects to determine which object will be associated with that location. 

Claim 21 (Previously Presented) A method as claimed in claim 14, wherein step h) includes the 
step of updating the stored attributes of the object as the attributes of the object change from 
frame to frame. 

Claim 22 (Previously Presented) A method as claimed in claim 14, wherein step h) includes the 
step of using plural algorithm means for calculation of independent tracks of objects for use 
depending on the visual complexity of a sequence automatically to track said objects in different 
types of visual environment. 

Claim 23 (Previously Presented) A method as claimed in claim 14, wherein step h) includes the 
steps of converting all the frames to be tracked to a low-level representation, determining the 
position of each object in the frames by minimising a distance measure to locate each object in 
each frame, processing the positions of said object to smooth over occlusions and the entrances 
and exits of objects into and out of said frames, reviewing the object within a tracked sequence 
and correcting the location attributes of any misplaced objects. 

Claim 24 (Previously Presented) A method as claimed in claim 14, wherein step g) includes the 
steps of providing a database of different types of data including one or more of URLs, HTML 
pages, video clips, audio clips, text files and multimedia catalogues, and selecting said 
interactive content data from the database to associate with said object. 

Claim 25 (Previously Presented) A method as claimed in claim 14, wherein step j) includes 
determining whether the embedded interactive content data is frame synchronous data associated 
with object positions, shapes, ranks and pointers in a frame, or group — synchronous data 
associated with all the objects in a group, or is data to be streamed just in time, and associating 
frame synchronous data with the corresponding frame, associating group synchronous data with 
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the frame at which a group changes, and streaming just in time data to a user before it is required 
to be associated with the corresponding objects, respectively. 

Claim 26 (Previously Presented) A method as claimed in claim 14, wherein in steps d) to j) 
different interactive content data are associated with respectively different objects. 

Claim 27 (Currently Amended) A computer readable medium embodying a computer program 
comprising code means for performing all the steps of the method of claim 14 when the program 
is run on one or more computers. 

Claim 28 (New) An apparatus for associating interactive data with an object in a digital 
video programme including means for providing a digital video programme having plural 
individual frames at least some of which incorporate said object, means for parsing the video 
programme by identifying separate shots in the video programme to produce an edit list, means 
for identifying shots containing related content to form a sequence of shots containing related 
content, means for selecting at least one key- frame within each sequence of shots, means for 
selecting said object within the key- frame with which the interactive data is to be associated, 
means for extracting attributes of the object from the key- frame, means for associating the 
interactive data with the object in the key- frame, means for utilising the attributes of the object 
for tracking the object through the sequence of shots, whereby said interactive data is associated 
with the object in the sequence of shots and said interactive data is embedded with data 
representative of said object in a data sequence, wherein said means for extracting attributes of 
said object includes means for comparing said attributes of said object with attributes of objects 
previously stored to determine whether the object is distinguishable therefrom, and when said 
object is determined not to be distinguishable, providing means for redefining the object. 

Claim 29 (New) An apparatus for associating interactive data with an object in a digital video 
programme including means for providing a digital video programme having plural individual 
frames at least some of which incorporate said object, means for parsing the video programme by 
identifying separate shots in the video programme to produce an edit list, means for identifying 
shots containing related content to form a sequence of shots containing related content, means 
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for selecting at least one key- frame within each sequence of shots, means for selecting said 
object within the key-frame with which the interactive data is to be associated, means for 
extracting attributes of the object from the key-frame, means for associating the interactive data 
with the object in the key-frame, means for utilising the attributes of the object for tracking the 
object through the sequence of shots, whereby said interactive data is associated with the object 
in the sequence of shots and said interactive data is embedded with data representative of said 
object in a data sequence, wherein said means for extracting said attributes includes means for 
comparing the location in the frame of said object with the location of objects already stored for 
that frame to determine whether that object is distinguishable therefrom, and where the location 
of said object is not distinguishable from the location of another object providing means for 
assigning rank to the objects to determine which object will be associated with that location. 

Claim 30 (New) An apparatus for associating interactive data with an object in a digital video 
programme including means for providing a digital video programme having plural individual 
frames at least some of which incorporate said object, means for parsing the video programme by 
identifying separate shots in the video programme to produce an edit list, means for identifying 
shots containing related content to form a sequence of shots containing related content, means 
for selecting at least one key-frame within each sequence of shots, means for selecting said 
object within the key-frame with which the interactive data is to be associated, means for 
extracting attributes of the object from the key-frame, means for associating the interactive data 
with the object in the key-frame, means for utilising the attributes of the object for tracking the 
object through the sequence of shots, whereby said interactive data is associated with the object 
in the sequence of shots and said interactive data is embedded with data representative of said 
object in a data sequence, wherein said tracking means utilising the attributes of the object for 
tracking the object includes plural algorithm means for calculation of independent tracks of 
objects for use depending on the visual complexity of a sequence to automatically track objects 
in different types of visual environment. 

Claim 3 1 (New) An apparatus for associating interactive data with an object in a digital video 
programme including means for providing a digital video programme having plural individual 
frames at least some of which incorporate said object, means for parsing the video programme by 
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identifying separate shots in the video programme to produce an edit list, means for identifying 
shots containing related content to form a sequence of shots containing related content, means 
for selecting at least one key- frame within each sequence of shots, means for selecting said 
object within the key-frame with which the interactive data is to be associated, means for 
extracting attributes of the object from the key-frame, means for associating the interactive data 
with the object in the key-frame, means for utilising the attributes of the object for tracking the 
object through the sequence of shots, whereby said interactive data is associated with the object 
in the sequence of shots and said interactive data is embedded with data representative of said 
object in a data sequence, wherein said tracking means for utilising the attributes of the object for 
tracking includes means for converting all the frames to be tracked to a low-level representation, 
means for determining the position of each object in the frames by minimising a distance 
measure to locate each object in each frame, means for processing the positions of said object to 
smooth over occlusions and the entrances and exits of objects into and out of said frames, and 
means for reviewing the object within a tracked sequence and for correcting the location 
attributes of any misplaced objects. 

Claim 32 (New) A method for associating interactive data representative of an object with a 
digital video programme including the steps of: 

a) providing a digital video programme having a plurality of individual frames at least some of 
which incorporate said object with which data is to be associated, 

b) parsing the video programme by identifying separate shots in the video programme to produce 
an edit list, 

c) identifying, from the edit list, shots containing related content to form a sequence of shots 
containing related content, 

d) selecting at least one key-frame containing the object from within the sequence of shots, 

e) locating said object within the at least one key-frame, 

f) extracting attributes of the object from the at least one key-frame including comparing said 
attributes of said object with attributes of objects previously stored to determine whether the 
object is distinguishable therefrom, and when said object is determined not to be distinguishable, 
re-defining the object, 

g) associating interactive data with the object in the at least one key-frame, 
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h) tracking the object through the sequence of shots utilising the attributes of the object, 

i) associating said interactive data with the object in frames in the sequence of shots, and 

j) embedding said interactive data with data representative of said object in a data sequence 
representative of the digital video programme. 

Claim 33 (New) A method for associating interactive data representative of an object with a 
digital video programme including the steps of: 

a) providing a digital video programme having a plurality of individual frames at least some of 
which incorporate said object with which data is to be associated, 

b) parsing the video programme by identifying separate shots in the video programme to produce 
an edit list, 

c) identifying, from the edit list, shots containing related content to form a sequence of shots 
containing related content, 

d) selecting at least one key-frame containing the object from within the sequence of shots, 

e) locating said object within the at least one key-frame, 

f) extracting attributes of the object from the at least one key-frame, including comparing the 
location in the frame of said object with the location of objects already stored for that frame to 
determine whether that object is distinguishable therefrom, and where the location of said object 
is not distinguishable from the location of another object, the step of assigning rank to the objects 
to determine which object will be associated with that location, 

g) associating interactive data with the object in the at least one key- frame, 

h) tracking the object through the sequence of shots utilising the attributes of the object, 

i) associating said interactive data with the object in frames in the sequence of shots, and 

j) embedding said interactive data with data representative of said object in a data sequence 
representative of the digital video programme. 

Claim 34 (New) A method for associating interactive data representative of an object with a 
digital video programme including the steps of: 

a) providing a digital video programme having a plurality of individual frames at least some of 
which incorporate said object with which data is to be associated, 
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b) parsing the video programme by identifying separate shots in the video programme to produce 
an edit list, 

c) identifying, from the edit list, shots containing related content to form a sequence of shots 
containing related content, 

d) selecting at least one key-frame containing the object from within the sequence of shots, 

e) locating said object within the at least one key-frame, 

f) extracting attributes of the object from the at least one key-frame, 

g) associating interactive data with the object in the at least one key-frame, 

h) tracking the object through the sequence of shots utilising the attributes of the object, 
including using plural algorithm means for calculation of independent tracks of objects for use 
depending on the visual complexity of a sequence automatically to track said objects in different 
types of visual environment, 

i) associating said interactive data with the object in frames in the sequence of shots, and 

j) embedding said interactive data with data representative of said object in a data sequence 
representative of the digital video programme. 

Claim 35 (New) A method for associating interactive data representative of an object with a 
digital video programme including the steps of: 

a) providing a digital video programme having a plurality of individual frames at least some of 
which incorporate said object with which data is to be associated, 

b) parsing the video programme by identifying separate shots in the video programme to produce 
an edit list, 

c) identifying, from the edit list, shots containing related content to form a sequence of shots 
containing related content, 

d) selecting at least one key-frame containing the object from within the sequence of shots, 

e) locating said object within the at least one key- frame, 

f) extracting attributes of the object from the at least one key-frame, 

g) associating interactive data with the object in the at least one key-frame, 

h) tracking the object through the sequence of shots utilising the attributes of the object, 
including converting all the frames to be tracked to a low-level representation, determining the 
position of each object in the frames by minimising a distance measure to locate each object in 



31 



each frame, processing the positions of said object to smooth over occlusions and the entrances 
and exits of objects into and out of said frames, reviewing the object within a tracked sequence 
and correcting the location attributes of any misplaced objects, 

i) associating said interactive data with the object in frames in the sequence of shots, and 
j) embedding said interactive data with data representative of said object in a data sequence 
representative of the digital video programme. 
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